{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "723fcf40",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/jerryjliu/llama_index/blob/main/docs/examples/retrievers/bm25_retriever.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bf1de44-4047-46cf-a04c-dbf910d9e179",
   "metadata": {},
   "source": [
    "# BM25 Retriever\n",
    "In this guide, we define a bm25 retriever that search documents using bm25 method.\n",
    "\n",
    "This notebook is very similar to the RouterQueryEngine notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e73fead-ec2c-4346-bd08-e183c13c7e29",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a2630bc",
   "metadata": {},
   "source": [
    "If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9bc13275",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install llama-index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a2d59778-4cda-47b5-8cd0-b80fee91d1e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# NOTE: This is ONLY necessary in jupyter notebook.\n",
    "# Details: Jupyter runs an event-loop behind the scenes.\n",
    "#          This results in nested event-loops when we start an event-loop to make async queries.\n",
    "#          This is normally not allowed, we use nest_asyncio to allow it for convenience.\n",
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04941476",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import openai\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = \"sk-...\"\n",
    "openai.api_key = os.environ[\"OPENAI_API_KEY\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c628448c-573c-4eeb-a7e1-707fe8cc575c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import sys\n",
    "\n",
    "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
    "logging.getLogger().handlers = []\n",
    "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n",
    "\n",
    "from llama_index import (\n",
    "    SimpleDirectoryReader,\n",
    "    ServiceContext,\n",
    "    StorageContext,\n",
    "    VectorStoreIndex,\n",
    ")\n",
    "from llama_index.retrievers import BM25Retriever\n",
    "from llama_index.indices.vector_store.retrievers.retriever import (\n",
    "    VectorIndexRetriever,\n",
    ")\n",
    "from llama_index.llms import OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d87b60f",
   "metadata": {},
   "source": [
    "## Download Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c9456c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p 'data/paul_graham/'\n",
    "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "787174ed-10ce-47d7-82fd-9ca7f891eea7",
   "metadata": {},
   "source": [
    "## Load Data\n",
    "\n",
    "We first show how to convert a Document into a set of Nodes, and insert into a DocumentStore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fc1b8ac-bf55-4d60-841c-61698663322f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load documents\n",
    "documents = SimpleDirectoryReader(\"./data/paul_graham\").load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7081194a-ede7-478e-bff2-23e89e23ef16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize service context (set chunk size)\n",
    "llm = OpenAI(model=\"gpt-4\")\n",
    "service_context = ServiceContext.from_defaults(chunk_size=1024, llm=llm)\n",
    "nodes = service_context.node_parser.get_nodes_from_documents(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f61bca2-c3b4-4ef0-a8f1-367933aa6d05",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize storage context (by default it's in-memory)\n",
    "storage_context = StorageContext.from_defaults()\n",
    "storage_context.docstore.add_documents(nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d6162df-9da7-4aad-a2ca-eb318f67daec",
   "metadata": {},
   "outputs": [],
   "source": [
    "index = VectorStoreIndex(\n",
    "    nodes=nodes,\n",
    "    storage_context=storage_context,\n",
    "    service_context=service_context,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fa4b18c-4f9b-4311-b1d9-9cbf24687e2c",
   "metadata": {},
   "source": [
    "## BM25 Retriever\n",
    "\n",
    "We will search document with bm25 retriever."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e94711d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install rank_bm25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee3f7c3b-69b4-48d5-bf22-ac51a4e3179f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can pass in the index, doctore, or list of nodes to create the retriever\n",
    "retriever = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b8c4c12-1a30-425e-8312-04be050b2101",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Node ID:** d95537b4-b398-4b47-94ff-da86f05a27f7<br>**Similarity:** 5.171801938898801<br>**Text:** I wanted to go back to RISD, but I was now broke and RISD was very expensive, so I decided to get...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** 6f84e2a5-1ab1-4389-8799-b7713e085931<br>**Similarity:** 4.838241203957084<br>**Text:** All you had to do was teach SHRDLU more words.\n",
       "\n",
       "There weren't any classes in AI at Cornell then, ...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from llama_index.response.notebook_utils import display_source_node\n",
    "\n",
    "# will retrieve context from specific companies\n",
    "nodes = retriever.retrieve(\"What happened at Viaweb and Interleaf?\")\n",
    "for node in nodes:\n",
    "    display_source_node(node)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2749c34e-97c0-4bd5-8358-377a94b8b2d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Node ID:** a4fd0b29-4138-4741-9e27-9f65d6968eb4<br>**Similarity:** 8.090884087344435<br>**Text:** Not so much because it was badly written as because the problem is so convoluted. When you're wor...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** d95537b4-b398-4b47-94ff-da86f05a27f7<br>**Similarity:** 5.830874349482576<br>**Text:** I wanted to go back to RISD, but I was now broke and RISD was very expensive, so I decided to get...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nodes = retriever.retrieve(\"What did Paul Graham do after RISD?\")\n",
    "for node in nodes:\n",
    "    display_source_node(node)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73460e63-3c7a-49f4-80d2-ea9f4033848e",
   "metadata": {},
   "source": [
    "## Router Retriever with bm25 method\n",
    "\n",
    "Now we will combine bm25 retriever with vector index retriever."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ceedab8-c9e8-4ec9-be91-cba27482a7e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.tools import RetrieverTool\n",
    "\n",
    "vector_retriever = VectorIndexRetriever(index)\n",
    "bm25_retriever = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=2)\n",
    "\n",
    "retriever_tools = [\n",
    "    RetrieverTool.from_defaults(\n",
    "        retriever=vector_retriever,\n",
    "        description=\"Useful in most cases\",\n",
    "    ),\n",
    "    RetrieverTool.from_defaults(\n",
    "        retriever=bm25_retriever,\n",
    "        description=\"Useful if searching about specific information\",\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dcb850f9-c8b4-4843-b66c-8cb2a977c0e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.retrievers import RouterRetriever\n",
    "\n",
    "retriever = RouterRetriever.from_defaults(\n",
    "    retriever_tools=retriever_tools,\n",
    "    service_context=service_context,\n",
    "    select_multi=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3e29b3e-38e3-4f0b-a263-d267ed003cc6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selecting retriever 0: The author's life context is a broad topic, which may require a comprehensive approach that is useful in most cases..\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** fcd399c1-3544-4df3-80a9-0a7d3fd41f1f<br>**Similarity:** 0.7942753162501964<br>**Text:** [10]\n",
       "\n",
       "Wow, I thought, there's an audience. If I write something and put it on the web, anyone can...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** b203e140-d549-4284-99f4-b1b5bcd996ea<br>**Similarity:** 0.7788031317604815<br>**Text:** Now all I had to do was learn Italian.\n",
       "\n",
       "Only stranieri (foreigners) had to take this entrance exa...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# will retrieve all context from the author's life\n",
    "nodes = retriever.retrieve(\n",
    "    \"Can you give me all the context regarding the author's life?\"\n",
    ")\n",
    "for node in nodes:\n",
    "    display_source_node(node)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1b4b717",
   "metadata": {},
   "source": [
    "## Advanced - Hybrid Retriever + Re-Ranking\n",
    "\n",
    "Here we extend the base retriever class and create a custom retriever that always uses the vector retriever and BM25 retreiver.\n",
    "\n",
    "Then, nodes can be re-ranked and filtered. This lets us keep intermediate top-k values large and letting the re-ranking filter out un-needed nodes.\n",
    "\n",
    "To best demonstrate this, we will use a larger set of source documents -- Chapter 3 from the 2022 IPCC Climate Report."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4be4ec7",
   "metadata": {},
   "source": [
    "### Setup data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ebd3ddf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100 20.7M  100 20.7M    0     0   361k      0  0:00:58  0:00:58 --:--:--  422k\n"
     ]
    }
   ],
   "source": [
    "!curl https://www.ipcc.ch/report/ar6/wg2/downloads/report/IPCC_AR6_WGII_Chapter03.pdf --output IPCC_AR6_WGII_Chapter03.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f7b5a97",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install pypdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e46f93b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index import (\n",
    "    VectorStoreIndex,\n",
    "    ServiceContext,\n",
    "    StorageContext,\n",
    "    SimpleDirectoryReader,\n",
    ")\n",
    "from llama_index.llms import OpenAI\n",
    "\n",
    "# load documents\n",
    "documents = SimpleDirectoryReader(\n",
    "    input_files=[\"IPCC_AR6_WGII_Chapter03.pdf\"]\n",
    ").load_data()\n",
    "\n",
    "# initialize service context (set chunk size)\n",
    "# -- here, we set a smaller chunk size, to allow for more effective re-ranking\n",
    "llm = OpenAI(model=\"gpt-3.5-turbo\")\n",
    "service_context = ServiceContext.from_defaults(chunk_size=256, llm=llm)\n",
    "nodes = service_context.node_parser.get_nodes_from_documents(documents)\n",
    "\n",
    "# initialize storage context (by default it's in-memory)\n",
    "storage_context = StorageContext.from_defaults()\n",
    "storage_context.docstore.add_documents(nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ccfc9879",
   "metadata": {},
   "outputs": [],
   "source": [
    "index = VectorStoreIndex(\n",
    "    nodes, storage_context=storage_context, service_context=service_context\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0474be06",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.retrievers import BM25Retriever\n",
    "\n",
    "# retireve the top 10 most similar nodes using embeddings\n",
    "vector_retriever = index.as_retriever(similarity_top_k=10)\n",
    "\n",
    "# retireve the top 10 most similar nodes using bm25\n",
    "bm25_retriever = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99e603a1",
   "metadata": {},
   "source": [
    "### Custom Retriever Implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "917a3d85",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.retrievers import BaseRetriever\n",
    "\n",
    "\n",
    "class HybridRetriever(BaseRetriever):\n",
    "    def __init__(self, vector_retriever, bm25_retriever):\n",
    "        self.vector_retriever = vector_retriever\n",
    "        self.bm25_retriever = bm25_retriever\n",
    "        super().__init__()\n",
    "\n",
    "    def _retrieve(self, query, **kwargs):\n",
    "        bm25_nodes = self.bm25_retriever.retrieve(query, **kwargs)\n",
    "        vector_nodes = self.vector_retriever.retrieve(query, **kwargs)\n",
    "\n",
    "        # combine the two lists of nodes\n",
    "        all_nodes = []\n",
    "        node_ids = set()\n",
    "        for n in bm25_nodes + vector_nodes:\n",
    "            if n.node.node_id not in node_ids:\n",
    "                all_nodes.append(n)\n",
    "                node_ids.add(n.node.node_id)\n",
    "        return all_nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9495e47c-2008-41ac-87e8-582d6d798190",
   "metadata": {},
   "outputs": [],
   "source": [
    "index.as_retriever(similarity_top_k=5)\n",
    "\n",
    "hybrid_retriever = HybridRetriever(vector_retriever, bm25_retriever)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfd3b287",
   "metadata": {},
   "source": [
    "### Re-Ranker Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1d910de",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install sentence_transformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "638e7786",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading (…)lve/main/config.json: 100%|██████████| 799/799 [00:00<00:00, 3.86MB/s]\n",
      "Downloading pytorch_model.bin: 100%|██████████| 1.11G/1.11G [00:32<00:00, 34.4MB/s]\n",
      "Downloading (…)okenizer_config.json: 100%|██████████| 443/443 [00:00<00:00, 2.19MB/s]\n",
      "Downloading (…)tencepiece.bpe.model: 100%|██████████| 5.07M/5.07M [00:00<00:00, 14.1MB/s]\n",
      "Downloading (…)cial_tokens_map.json: 100%|██████████| 279/279 [00:00<00:00, 1.48MB/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Use pytorch device: cpu\n"
     ]
    }
   ],
   "source": [
    "from llama_index.postprocessor import SentenceTransformerRerank\n",
    "\n",
    "reranker = SentenceTransformerRerank(top_n=4, model=\"BAAI/bge-reranker-base\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58a78eec",
   "metadata": {},
   "source": [
    "### Retrieve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9235f79d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Batches: 100%|██████████| 1/1 [00:05<00:00,  5.61s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial retrieval:  19  nodes\n",
      "Re-ranked retrieval:  4  nodes\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "from llama_index import QueryBundle\n",
    "\n",
    "nodes = hybrid_retriever.retrieve(\n",
    "    \"What is the impact of climate change on the ocean?\"\n",
    ")\n",
    "reranked_nodes = reranker.postprocess_nodes(\n",
    "    nodes,\n",
    "    query_bundle=QueryBundle(\n",
    "        \"What is the impact of climate change on the ocean?\"\n",
    "    ),\n",
    ")\n",
    "\n",
    "print(\"Initial retrieval: \", len(nodes), \" nodes\")\n",
    "print(\"Re-ranked retrieval: \", len(reranked_nodes), \" nodes\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5674f1d6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Node ID:** 74b12b7b-f4b9-490a-9342-b640211468dd<br>**Similarity:** 0.998129665851593<br>**Text:** 3\n",
       "469Oceans and Coastal Ecosystems and Their Services  Chapter 3\n",
       "Frequently Asked Questions\n",
       "FAQ 3...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** 2b35824c-2e96-47b7-8dfb-da25c4eefb7d<br>**Similarity:** 0.996731162071228<br>**Text:** {Box 3.2, 3.2.2.1, 3.4.2.5, 3.4.2.10, 3.4.3.3, Cross-Chapter \n",
       "Box PALEO in Chapter 1}\n",
       "Climate imp...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** 01ef2a9e-0dd0-4bce-ab60-e6a3f6456f7b<br>**Similarity:** 0.9954373240470886<br>**Text:** These ecosystems are also influenced by non-climate drivers, especially fisheries, oil and gas ex...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**Node ID:** 8a23b728-0352-4b01-a5c0-42765669855d<br>**Similarity:** 0.9872682690620422<br>**Text:** Additionally, climate-change-driven oxygen loss (Section  3.2.3.2; Luna et  al., 2012; \n",
       "Belley et...<br>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from llama_index.response.notebook_utils import display_source_node\n",
    "\n",
    "for node in reranked_nodes:\n",
    "    display_source_node(node)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c24caa0d",
   "metadata": {},
   "source": [
    "### Full Query Engine "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "549b9a96",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Batches: 100%|██████████| 1/1 [00:05<00:00,  5.74s/it]\n"
     ]
    }
   ],
   "source": [
    "from llama_index.query_engine import RetrieverQueryEngine\n",
    "\n",
    "query_engine = RetrieverQueryEngine.from_args(\n",
    "    retriever=hybrid_retriever,\n",
    "    node_postprocessors=[reranker],\n",
    "    service_context=service_context,\n",
    ")\n",
    "\n",
    "response = query_engine.query(\n",
    "    \"What is the impact of climate change on the ocean?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "171668b7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**`Final Response:`** Climate change has significant impacts on the ocean. It is degrading ocean health and altering stocks of marine resources. This, combined with over-harvesting, is threatening the sustenance provided to Indigenous Peoples, the livelihoods of artisanal fisheries, and marine-based industries such as tourism, shipping, and transportation. Climate change can also influence human activities and employment by altering resource availability, spreading pathogens, flooding shorelines, and degrading ocean ecosystems. Additionally, increases in intensity, reoccurrence, and duration of marine heatwaves due to climate change can lead to species extirpation, habitat collapse, and surpassing ecological tipping points. Some habitat-forming coastal ecosystems, including coral reefs, kelp forests, and seagrass meadows, are at high risk of irreversible phase shifts due to marine heatwaves. Non-climate drivers such as fisheries, oil and gas extraction, cable laying, and mineral resource exploration also influence ocean ecosystems."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from llama_index.response.notebook_utils import display_response\n",
    "\n",
    "display_response(response)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llama-index-4a-wkI5X-py3.11",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
